|
1
|
- My search for the missing link between mind and body –
- Rainer von Königslöw, Ph.D.-
- Presented at University of Guelph on Oct. 14, 2008-
|
|
2
|
- To model how the brain manages action
- Using a feasibility study with a top-down design
- With two types of solo learning
- To model the integration of perception into action
- Including conditional action and prediction
- With social learning like mimicry and imprinting
- To demonstrate how this paradigm, based on information processing sy=
stem
analysis, can shed light on the evolution of language and learning
within vertebrates
|
|
3
|
|
|
4
|
- I propose a new research paradigm, based on the approach typically u=
sed
for designing and building complex information-processing systems
- Use a top-down requirements analysis
- Develop an overall system architecture
- Use feasibility studies, supported by simulation models
- Look at capacity, performance, fault tolerance, etc.
|
|
5
|
- Walking is a good example
- Start with an analysis of system components and functional requireme=
nts
- Simplification is a key aspect of this approach, so we start with a
stick figure with bones and joints
- We start by analyzing a single stride
|
|
6
|
- Here is the highly simplified stick-figure model of the skeleton tha=
t we
use in the research
- We use balls for the joints
- We use cylinders for the limbs
- We use rectangular boxes for the head, shoulders, hips, hands, and f=
eet
|
|
7
|
- Move the bones by rotating joints
- Coordinate the movements with appropriate timing
- Use polar coordinates for each joint
- Front: the normal plane of rotation
- Side: sideways rotation e.g. for the hip-joint and the shoulder-joi=
nt
- Axis: rotating the bone
|
|
8
|
|
|
9
|
- Alternatives:
- Hard-wired information processing
- ENIAC, Wang, pre-microprocessor controls
- Stored-instruction and the von Neumann architecture
- Most modern system from mainframes through PCs to embedded controll=
ers
use software or firmware
- Need a hardware and software architecture
- For software, have instructions and data
- Computational equivalence
|
|
10
|
|
|
11
|
|
|
12
|
- SCADA: Supervisory Con=
trol
And Data Acquisition
- Process control with actuators, sensors, and a controller
- Muscles need constant input, not just change information
- For simplification and to allow easy monitoring, we shall assume
synchronous input at 30msec intervals (30 frames/ sec for video
recordings)
|
|
13
|
|
|
14
|
|
|
15
|
|
|
16
|
|
|
17
|
- Use multiple layers in large application
- Separate presentation, business logic, database access
- May even separate access to the hardware to make the application
machine independent
- Distribute the computing to different locations and devices
- Improve performance and fault tolerance
- Reuse computing facilities, both hardware and software
- Compress code, reduce memory usage, more maintainable
|
|
18
|
- Reusable subroutines date back to early days of software development=
- Reusability has taken many forms, e.g. object oriented programming
- Reusability of hardware components has been an issue right from the
ENIAC on
- The ENIAC used a standard triode
- NAND and NOR logic and flip-flops
- Standardized ICs including clocks, counters, processors
|
|
19
|
|
|
20
|
- Discussed: possible brain architectures for manage information
processing for action
- all of these architectures are based directly or indirectly on the =
von
Neumann model with its concept of stored instructions, i.e. softwar=
e
- Next: develop a concept of software and programming for the brain
- Programming depends on programming languages, ranging from machine =
code
through assembler, Fortran, C to 3rd generation languages
like Lisp and Prolog
|
|
21
|
- Like ballet choreography for providing detailed specifications for p=
oses
and movements
- Includes stage directions
- Uses macro expansion like compilers and interpreters
|
|
22
|
|
|
23
|
- Most physical actions such as walking take place in a 3D geometric
space, with a floor and gravity – a ‘world’ space<=
/li>
|
|
24
|
- Joint rotations are in frames that are relative to the supporting bo=
ne,
with a separate frame for each joint
- E.g. hip -> hip-joint -> upper leg -> knee -> lower leg
-> ankle -> foot
- Polar coordinates, up to 3D
- Instructions to the muscles have to be in the local frame for the jo=
int
|
|
25
|
- Discussed: how the brain might manage information processing for act=
ion
- Developed an overall system architecture for both hardware and soft=
ware
- For both programming instructions and data representations
- Next, back to requirements: show how the architecture might be used =
to
support simple learning
- After that: show how the brain might integrate vision into the
information processing to improve action
|
|
26
|
- Trial and error learning
- Somewhat random generation of action
- A language lends itself to substitutions
- Selection on success criteria
- Use ‘satisficing’ rather than optimizing
- Remember and execute selectively
- Design and evaluate potential actions
- Evaluate instructions without execution
- The evaluation requires prediction
- This approach does not require possibly fatal errors for learning<=
/li>
- This approach is not as inherently unstable as one trial learning
(with superstitious learning, etc.)
|
|
27
|
- Examples:
- Random uncoordinated motions to coordinated motions such as crawling
and walking
- Play activities – across many species
- Babbling to speech
- Mechanism:
- Inner language ‘sentences’ generated by inner grammars =
lend
themselves to random substitution of sentence elements for the
generation of alternative actions
|
|
28
|
- Visual input for a single eye is in 2D and is pixelated
- The visual input is relative to the position of the eye
- The visual input is the response to an action that includes neck
rotation and eye rotation
|
|
29
|
- At the level where visual information is integrated with the geometr=
ic
component of action instructions, we propose a 3D representation that
can be scaled, rotated, and translated
- Scalable vector graphics (SVG) is an open 2D standard on the Web
- There is as yet no standard for modeling 3D objects (wiremesh or so=
lid)
|
|
30
|
- Conditionality is required so that visual information can be used to
select the best of two or more action sequences
|
|
31
|
- Showed how the brain might manage information processing for action =
with
integrated vision
- Developed an overall system architecture for both hardware and soft=
ware
- For both programming instructions and data representations
- Next, back to requirements: show how the information might be used to
support prediction and learning
|
|
32
|
|
|
33
|
|
|
34
|
|
|
35
|
- Common across vertebrates
- Vision-based comparison to adjust action
- Requires a 3D model of self
- Requires a 3D model of the person to be imitated
- 3D models are scaled and transformed so that they can be superimpos=
ed
so that differences in bone angles can be detected
- Instructions have to be generated and sent to the appropriate joint=
s to
correct angles
|
|
36
|
- There are quite a few behaviour patterns the require the integration=
of
visual information and conditional instructions for action
- Having the adults face the threat with young ones behind them, migh=
t be
one such example
|
|
37
|
|
|
38
|
- Common across vertebrates
- Vision-based comparison to adjust action
- Requires a 3D model of self
- Requires a 3D model of the person to be imitated
- 3D models are scaled and transformed so that they can be superimpos=
ed
so that differences in bone angles can be detected
- Instructions have to be generated and sent to the appropriate joint=
s to
correct angles
|
|
39
|
- Social learning: apprenticeship & mimicry
- Behaviour is observed, copied, and stored
|
|
40
|
- Potential actions can be aborted or modified because of predictions =
that
in turn might be based on visual information
- This kind of rudimentary reasoning appears to apply to many species=
- Inner language based expressions lend themselves to supporting this
kind of reasoning
|
|
41
|
- Showed how the brain might manage information processing for action<=
/li>
- Showed how the brain might integrate vision into the information
processing to improve action
- Showed how the information might be encoded to support prediction and
learning
- Next: status of the research
- Finally, speculate how the inner language might be utilized & le=
ad
to an outer language
|
|
42
|
- I have built a sequence of feasibility model showing actions such as
walking, sitting on a chair, and jumping – based on scripts
without visual input or conditionality
- I am presently working on integrating visual information to allow
conditional action
- The present model assumes that the 2D pixelated image has been
translated into a 3D vector representation. The focus is on geometric
transformations to guide the action, such as in mimicry and on walk=
ing
to and sitting on a chair.
|
|
43
|
- There are ongoing challenges in modeling timing coordination, especi=
ally
in the lower layers which are assumed to function quite independentl=
y
- Timing lags, such as from vision to action, flow naturally from the
model
- There are a number of feedback phenomena that can be explored once =
the
model is more complete and functioning more reliably
|
|
44
|
- I have tried to show how the inner language might be used to make
predictions such as for path planning by predators
- There is certainly evidence of conditionality and simple reasoning e=
.g.
in fight or flight
- I am still focused on making the basic model work, but I would
appreciate input and comments
|
|
45
|
- Human language is based on action
- Speech and writing are physical acts
- Reading requires vision.
Systematic speed reading is trained in conjunction with movi=
ng a
finger, but suppressing sub-vocal speech
- Listening and understanding speech involves attention and focus, and
often involves positioning the head. However, I am not sure that
analysis as a physical act would help.
|
|
46
|
- I have modeled the inner language as being composed of action
instructions and geometric components
- This corresponds roughly with verb phrases and with noun phrases in a
transformational grammar
- Editing the inner language might lead to talking to oneself which mi=
ght
lead to speech
|
|
47
|
- After some decades of knowledge engineering across most sectors of
society, I feel that there is no ‘universal semantic
space’. There is=
no
universally shared reality.
- Meaning is shared within professions and by people who are engaged in
common tasks
|
|
48
|
- I see language as enabling and supporting more and more complex and
cooperative tasks
- Externalization of the action component of the inner language might =
be
related to threatening behaviours, etc.
- I am not aware of many illustrations of the externalization of the
geometric component of the inner language
|
|
49
|
- Use of an inner software allows better and more efficient use of
hardware components
- Zero trial – predictive learning can be more stable and less
costly than 1, 2, or 3 trial learning
- More complex tasks can be encoded
|
![](3D"notes_flag.gif")
No=
tes![](3D"buttons.gif")
![](3D"slide0008_image001.png"){kind=link}
![](3D"slide0008_image002.gif"){kind=link}
![](3D"slide0010_image003.png"){kind=link}
![](3D"slide0010_image004.gif"){kind=link}
![](3D"slide0010_image005.png"){kind=link}
![](3D"slide0010_image006.gif"){kind=link}
![](3D"slide0010_image007.png"){kind=link}
![](3D"slide0010_image008.gif"){kind=link}
![](3D"slide0010_image009.png"){kind=link}
![](3D"slide0010_image010.gif"){kind=link}
![](3D"slide0005_image011.png"){kind=link}
![](3D"slide0005_image012.jpg"){kind=link}
![](3D"slide0006_image013.png"){kind=link}
![](3D"slide0006_image014.jpg"){kind=link}
![](3D"slide0012_image015.png"){kind=link}
![](3D"slide0012_image016.jpg"){kind=link}
![](3D"slide0011_image017.png"){kind=link}
![](3D"slide0011_image018.gif"){kind=link}
![](3D"slide0013_image019.png"){kind=link}
![](3D"slide0013_image020.gif"){kind=link}
![](3D"slide0014_image021.png"){kind=link}
![](3D"slide0014_image022.gif"){kind=link}
![](3D"slide0016_image023.gif"){kind=link}
![](3D"slide0018_image024.png"){kind=link}
![](3D"slide0018_image025.jpg"){kind=link}
![](3D"slide0023_image026.png"){kind=link}
![](3D"slide0023_image027.gif"){kind=link}
![](3D"slide0023_image028.png"){kind=link}
![](3D"slide0023_image029.gif"){kind=link}
![](3D"slide0023_image030.png"){kind=link}
![](3D"slide0023_image031.gif"){kind=link}
![](3D"slide0023_image032.png"){kind=link}
![](3D"slide0023_image033.gif"){kind=link}
![](3D"slide0024_image034.png"){kind=link}
![](3D"slide0024_image035.gif"){kind=link}
![](3D"slide0025_image036.png"){kind=link}
![](3D"slide0025_image037.jpg"){kind=link}
![](3D"slide0029_image038.png"){kind=link}
![](3D"slide0029_image039.jpg"){kind=link}